A study of diagrammatic ink in lecture

In this paper, we present a study of how instructors draw diagrams in the process of delivering lectures. We are motivated by wanting to understand challenges and opportunities for automatically analyzing diagrams, and to use this to improve tools to support the delivery of presentations and the viewing of archived lectures. The study was conducted by analyzing a large group of examples of diagrams collected from real lectures that were delivered from a Tablet PC. The main result of the paper is the identification of three specific challenges in analyzing spontaneous instructor diagrams: separating the diagram from its annotations and other surrounding ink, identifying phases in discussion of a diagram, and constructing the active context in a diagram. 1 Background and Study Details Delivery of presentations using electronic tools is becoming prevalent. Advantages of electronic delivery include the high quality of displayed materials, ease of reuse, and the ability to share materials across machines and archive artifacts of the presentation. The technology to support presentation is rapidly advancing, in particular, there is growing use of digital ink with electronic slides. In many situations, the ability to draw spontaneous diagrams to support exposition greatly enhances communication. In this paper we explore Email addresses: [email protected] (Richard Anderson), [email protected] (Ruth Anderson), [email protected] (Crystal Hoyer), [email protected] (Craig Prince), [email protected] (Jonathan Su), [email protected] (Fred Videon), [email protected] (Steven Wolfman). Preprint submitted to Elsevier Science 5 January 2005 properties of diagrams that are drawn naturally while delivering lectures. Our interest is to gain an understanding of drawing practices to inform the development of improved tools to support electronic presentations. Our domain of study is university lectures delivered by an instructor writing on electronic slides with digital ink on a Tablet PC. We collected data using Classroom Presenter (Anderson et al., 2004a), a system that we developed. However, there are many other systems that provide similar functionality. These include university developed systems such as eFuzion (Peiper et al., 2004) and DyKnow (Berque et al., 2004) and commercial applications such as Microsoft’s PowerPoint, OneNote and Journal. Basic support for digital ink in lecture is provided by all of these systems — high quality ink over lecture slides displayed in real time to the audience — but these systems could all benefit from automated analyses of ink that enhance the in-class experience or interaction with archived materials. For example, identifying and removing ink with transient meaning would make it easier for students to view the ink with persistent, important meaning. Similarly, selectively rerendering the most critical ink to improve legibility (e.g., by exploiting spoken or displayed context) would aid students’ learning. Once recognized, text and diagrams may also be rerendered to make them accessible for students with limited vision. Tidying messy text and simplifying diagrams would facilitate student note taking, both by removing extraneous detail and by freeing up space for students’ own annotations. Together these and other analyses could be employed off-line to transform the live lecture experience into a static, browsable summary suitable for review (e.g., to be posted online after the lecture). Indexing and search are key use cases for both static summaries and standard archives. All of these operations require understanding instructors’ ink. Before pursuing automatic analysis of this ink, we wanted to understand the types of inking that occur. In previous work (Anderson et al., 2004c), we identified three key types of inking that instructors employ: “attentional” ink that ties spoken utterance to slide content (e.g., the underlines and circles shown in Figure 1), standard textual annotations, and diagrammatic ink. We have already explored styles of use and strategies for understanding attentional and textual ink (Anderson et al., 2004b). In this study, we address the key remaining annotation type by investigating how instructors use digital ink to construct diagrams in lectures. We examined a large number of diagrams that occurred naturally in university lectures, with the goal of identifying general patterns of diagrammatic writing that would indicate challenges and opportunities for automatic analysis. In this paper we identify three key phenomena we have observed in instructors’ use of diagrammatic ink during lecture. Our observations are from actual